The present invention is directed to an apparatus for processing sound and image data, and more particularly to a game computer for displaying image information and producing accompanying sound.
A conventional computer will be explained, hereinafter, separately image processing and sound processing.
(1) IMAGE PROCESSING
In a conventional game computer, a background (BG) image and a sprite image moving on the BG are combined on a screen to increase processing speed and to lower cost.
The background image is composed of a plurality character blocks (character screens) each defined by 64 dots of 8-by-8. Therefore, the background image is defined on a CRT display screen by deciding position, color and pattern of the character blocks. The computer has a video-RAM (VRAM) which includes a background attribute table (BAT) and a character generator (CG), as shown in FIG. 1. The background image is defined in accordance with data format of the background attribute table (BAT) and the character generator (CG).
The BAT includes a CG COLOR and a character code whereby position and color for each character are defined on a virtual screen, which is larger in size than the real screen (CRT). The CG stores many actual character patterns corresponding to the character codes in the BAT. The CG is, for example, composed of four character elements CH0 to CH3, and the number of the character elements is defined by color modes. For instance, the CG is composed of two character elements in a four-color mode, and is composed of four character elements in a sixteen-color mode.
The color of each character block is defined dot by dot, each dot color is defined by total bits of all the corresponding dots on each character elements CH0 to CH3, as shown in FIG. 2. Specifically, when the corresponding dots of character elements CH0 to CH3 are indicated by b0, b1, b2 and b3, a color "C" of the dot to be represented is given by an equation "C=b0.times.2.sup.0 +b1.times.2.sup.1 +b2.times.2.sup.2 +b3.times.2.sup.3 ". It can be considered that the color "C" may be treated as a color code directly. However, the conventional game computer uses a color pallet which stores plural color codes to manage colors of the background image so that many colors may be used for displaying one background image. The color pallet is indicated in position by the color codes of the CG.
The character code in the BAT indicates an address in the CG. A color to be displayed is selected from colors in the color pallet in accordance with both the CG COLOR and the color code. That is, first, a color block is selected from the CG in accordance with the CG COLOR, then a color is selected from in the color block in accordance with the color code, as shown in FIG. 3. The CG COLOR is composed of 4 bits, so that sixteen color blocks may be represented thereby. Each color code is composed of 4 bits, whereby sixteen colors may be represented for each character. Ultimately, sixteen colors are selected from among 256 (16.times.16) colors.
When an address of the image data is moved in a horizontal or vertical direction on the virtual screen, the image is scrolled on the CRT display.
Generally, transmission process of image data to the CRT is controlled in accordance with horizontal and vertical synchronizing signals. According to an NTSC system, 525 scanning lines are used to display an image, and odd fields and even fields are scanned alternately.
On the display device for a general game computer, only odd or even fields are scanned. The display system is controlled line by line in synchronization with horizontal synchronizing signals, and is controlled screen by screen in synchronization with vertical synchronizing signals. In the vertical synchronizing signals, there are flyback times longer than those of the horizontal synchronizing signals, whereby it is preferable that the image data are processed in the flyback times of the vertical synchronization. In order to display the BG image on the CRT, a raster scanning position is detected, and then each piece of character information is converted into video signals.
It is possible to realize an image rotation process and a synthesis process of plural images. In the rotation process of the BG screen, a plurality of screens having different display angles are displayed in a predetermined order whereby the BG screen seems as if it is rotated. In the other hand, a plurality of screens having different angles, which are produced by calculation according to a predetermined matrix coordinate, are displayed in the same order. In the synthesis process, a plurality of image data are supplied through different buses to an encoder, and the data are mixed by the encoder.
The conventional computer system includes a fader for giving a 100% brightness to a picture to be displayed, and a 0% brightness to the other pictures.
Most conventional game computers have single screen, that is not changed by another.
(2) SOUND PROCESSING
In general, a computer has a digital sound source in which all of sound signals are indicated as numerals whereby synthesis of waveform is carried out by addition, subtraction, multiplication and division calculations. Most of game computers include programmable sound generators (PSG) each manage a small amount of data. According to the PSG, waveform data in a predetermined period are changed in amplitude, then modulated in frequency and then are processed in order to generate a sound waveform. In the PSG, the sound source (sound waveform) is generated in accordance with a predetermined program.
On another way, analog sound data are converted into digital sound data whereby high quality sound source is generated by a pulse code modulation (PCM) method. According to the PCM, an analog signal is sampled at predetermined intervals, the sampled signal is quantified and is converted into binary number, whereby digital data are generated.
For the game computer, an adaptive-difference pulse code modulation (ADPCM) method is used for generating a digital signal. According to the ADPCM method, the difference between next two sample values is quantified, and the sampling pitch is shortened if the quantified difference value is larger, and is extended if the quantified difference value is smaller, whereby data to be used for a sound source are reduced (scale-down). As a result, sound can be reproduced by using a small amount of data. The PCM data and ADPCM data may be converted to each other in accordance with a scale coefficient and an extension coefficient given by a scale value and a scale level.
In the ADPCM method, high quality sound may be produced when a sampling frequency of a scale rate is large. In the game computer, sampling frequency is determined on the basis of the data amount, the maximum value being about 16 kHz.
In operation, the ADPCM data are read from the memory by the CPU, and the scale value and the scale level of the data are detected by an ADPCM decoder. The ADPCM data are extended to PCM data, whereby sound is reproduced at a reproducing rate corresponding to the sampling frequency. The reproducing rate is controlled by a synchronizing signal generating circuit including a decoder.
Address renewal of the RAM is carried out by the CPU in accordance with a predetermined program. As another way, such address renewal is carried out in accordance with an automatic adding process. For example, if data access is performed for 32 blocks at each time, the first address of the data are specified and then the following addresses are specified by adding predetermined values to the previous address in the same order. Therefore, it is realized that the RAM is accessed continuously or accessed with some interval depending on the added value.
As described above, the conventional game computer uses BG image data generally composed of only external block sequential data, each block being indicated by 8-by-8 dots, the image is displayed by an RGB system. A variety of image data such as a natural picture illustrated with continuous colors, a moving picture and a still picture illustrated by a single color are required to be displayed, however, the conventional game computer can display only a simple image. Especially, according to the conventional technology, it is difficult to display the natural picture when the CPU has a large amount of data have to deal with.
If the conventional computer having a simple RAM deals with data other than block sequential type data, the processing speed becomes low.
The computer is required to deal with a large amount data in order to display the natural picture. When each luminance of the RGB is defined by eight bits on one screen having picture elements of 512.times.512, each color image needs a memory capacity of about 768 k bytes (512.times.512.times.3). Therefore, it is necessary to take full advantage of the RAM in order to keep the processing speed high when a large amount data have to be processed. It is easy to deal with such a large amount data if all the different mode data are arranged in the same manner in the RAM, however, effective use of the RAM can not be realized. If the image data are arranged in accordance with the mode thereof, different data accesses are required for each mode, so that the address process becomes complicated. Therefore, a memory access apparatus which can manage the RAM having a complicated data arrangement is required for the computer.
In the game computer, predetermined data must be processed while tens of images are displayed within a second, and therefore processing must be performed in both horizontal (HSYNC) and vertical synchronizing (VSYNC) periods. For that reason, in the conventional game computer, it is difficult to realize special graphic processes such as rotation, magnification, reduction and the like. Further, the conventional game computer uses a fader to synthesize a plurality of BG images, so that the structure is complicated and the CPU must perform much processing in the case where a variety of image data are processed and large number of BG images are synthesized. Although BG image data may be processed in both the HSYNC and VSYNC periods for single BG image of 8.times.8 blocks, it is difficult to avoid the disadvantages when a variety of image data are processed or large number of BG data are processed.
As for sound data processing, the conventional ADPCM has a sample frequency of 16 kHz maximum, so that sound quality is not always high. If the sample frequency is increased for producing high quality sound, data amount to be processed is increased. As a result, the above mentioned problems, namely, low speed processing and complicated data access become remarkable, if image data are treated together with PSG data and ADPCM data.